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FIELD OF THE INVENTION 
[0001] The present invention relates generally to an image capturing device, 
and more particularly to real time noise characterization and reduction in an 
electronic image sensor of an image capturing device. 

BACKGROUND OF THE INVENTION 
[0002] Image capturing devices are commonly used to capture scenes, 
persons, events, settings, etc. Image capturing devices may come in a variety of 
types including, for example, digital still cameras and video cameras. A typical 
image capturing device includes a lens and an electronic image sensor, such as a 
D charge-coupled device (CCD) or complementary metal oxide semiconductor (CMOS) 

^ array. The electronic image sensor comprises a two-dimensional array of light 

y sensitive pixel elements that produce electronic signals in response to light. The 

' electronic image sensor therefore outputs electronic signals that are related to the 
jn amount of light impinging on the individual pixel elements of the array. 

l'^ [0003] One problem encountered with an electronic image sensor is noise that 

^ is picked up during an image capturing process. In addition to creating an output 

signal based on light impinging on the sensor elements, the sensor array may also 
create output signals based on noise, where the noise is generated by unwanted 
light or generated due to thermal effects or stray currents in the electronic image 
sensor. This noise corrupts a captured image and therefore is highly undesirable. 
[0004] A first prior art noise reduction approach involves dark frame subtraction, 
wherein the prior art camera captures a regular image and then captures a dark 
image. The dark image is captured at about the same exposure time and generally 
is captured when the shutter is closed, i.e., when the electronic image sensor is dark. 
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The prior art approach thereafter subtracts the dark image information from the 
regular image information in order to remove the noise from the regular image. This 
improves the overall image captured by the image capturing device. 
[0005] However, this prior art approach has several drawbacks. Capturing a 
dark image after a regular image requires an exposure for each image capture {i.e., 
a double exposure). This Increases shot-to-shot time, and moreover increases 
processing time. This processing time requires a time component for capturing the 
dark image and a time component for subtracting the dark image. 
[0006] A second prior art noise reduction approach employs a fixed dark cell in 
the camera. This dark cell may be a pixel element that does not participate in an 
image capture, but instead is a pixel element that is sealed off or covered and cannot 
generate anything but a dark reading {i.e., a blind cell). The dark cell gives an 
average or approximate noise reading for the entire electronic image sensor and 
does not give a reading for each individual pixel element. This average noise value 
is then subtracted from all pixels of the captured image. This prior art approach is 
much faster than a full dark image capture, and does not require a dark image 
capture after the regular image capture, i.e., the dark cell may obtain a reading 
concurrently with the image capture. 

[0007] However, there are drawbacks in the second prior art approach. The 
dark cell is less effective at noise reduction. The dark cell output may not accurately 
reflect the varying noise levels in each individual pixel element. The dark cell 
approach therefore does not take into account variations in pixel elements and 
moreover becomes less accurate over long exposure periods where pixel-to-pixel 
noise variations can be significant. 
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[0008] Therefore, there remains a need in the art for improvements in noise 
reduction for electronic image sensors. 

SUMMARY OF THE INVENTION 
[0009] An image capturing device comprises an electronic image sensor and a 
memory including a dark frame buffer that stores one or more darl< frames generated 
by the electronic image sensor. A processor controls the electronic image sensor to 
substantially continuously capture and store a newest dark frame from the electronic 
image sensor when the electronic image sensor is not performing an image capture. 
The processor subtracts the newest dark frame from an image upon an image 
£1 capture. 

ri BRIEF DESCRIPTION OF THE DRAWINGS 

fU [0010] FIG. 1 is a schematic diagram of an image capturing device according to 

m one embodiment of the invention; 

[001 1] FIG. 2 is a flowchart of a noise reduction method according to one 
O embodiment of the invention; and 

[0012] FIG. 3 is a flowchart of a noise reduction method according to another 

embodiment of the invention. 

DETAILED DESCRIPTION 
[0013] FIG. 1 is a schematic diagram of an image capturing device 100 
according to one embodiment of the invention. The image capturing device 100 
includes a lens apparatus 102, an electronic image sensor 105, a processor 109, 
and a memory 120. 
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[0014] The electronic image sensor 105 may be any type of electronic image 
sensor that comprises an array of individual pixel elements. For example, the 
electronic image sensor 105 may be a charge-coupled device (CCD) or 
complementary metal oxide semiconductor (CMOS) array. 
[0015] The memory 120 may be any type of digital memory. The memory 120 
may include, among other things, a dark frame buffer 123, a plurality of images 126, 
a preview mode variable 132, a capture mode variable 136, a dark frame preview 
timer 138, and a capture timer 150. In addition, the memory 120 may store software 
or firmware to be executed by the processor 109. 

[0016] The processor 109 may be any type of general purpose processor. The 
processor 109 executes a control routine contained in the memory 120. In addition, 
the processor 109 receives inputs and conducts image capturing, image 
manipulating and processing, and image storing processes. 
[0017] The dark frame buffer 1 23 is a circular buffer capable of holding one or 
more captured dark frames. A dark frame is a set of values for all pixel elements in 
the electronic image sensor 105. The one or more dark frames may be stored for a 
period of time. The period of time depends on the size of the circular buffer. 
Therefore, the dark frame buffer 123 essentially comprises a continuous loop 
wherein as a newest dark frame is stored, it replaces an oldest dark frame in the 
dark frame buffer 123. Consequently, at any point in time, the dark frame buffer 123 
contains a substantially current dark frame. 

[0018] The capture mode variable 136 is a variable that indicates when the 
image capturing device 100 is engaged in an image capture operation, such as a 
shutter operation, for example. The capture mode variable may be set in response 
to a shutter button press. 
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[0019] A dark frame may be captured only when the shutter of the image 
capturing device 100 is closed. Therefore, when the image capturing device 100 is 
not capturing images, it may capture dark frames and store them to the dark frame 
buffer 123. 

[0020] The dark frame data contained within the dark frame buffer 123 may be 
used whenever an image capture occurs. Therefore, upon an image capture, the 
processor 109 may recall the most current dark frame from the dark frame buffer 123 
and subtract it from the captured image, with the captured image then being stored, 
such as in the image storage 126. 

[0021] In operation, the processor 109 may substantially continuously initiate 
the capture of dark frames and store them in the dark frame buffer 123. Therefore, 
for example, a new dark frame capture may be performed substantially immediately 
upon completion of a previous dark frame capture. Alternatively, a sliding average 
may be calculated and maintained, wherein a newly captured dark frame is 
calculated into an existing dark frame average. The sliding average therefore may 
be an average of a predetermined number of recently acquired dark frames or may 
be an average of all dark frames acquired within a predetermined time period. 
[0022] As a result, upon an image capture, a substantially current dark frame is 
available. This substantially current dark frame characterizes Image noise and may 
be subtracted from the image in order to reduce the noise in the image. The image 
is thereby greatly improved, and without the dark frame capture unduly interfering 
with the image capturing process. 

[0023] Altematively, the image capturing device 100 may periodically capture 
dark frames, such as upon a predetermined time period. For example, the 
predetermined time period may be loaded into the capture timer 150. When the 
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capture timer 150 expires, a dark frame may be captured, and the capture timer 150 
is reloaded with the predetermined time period. In this manner, the dark frames may 
be captured in a substantially continuous manner, but yet without consuming 
excessive processor time and without consuming excessive electrical power, 
[0024] The preview mode variable 132 is a variable that indicates whether the 
image capturing device 100 is in a preview mode. A preview mode is a mode 
wherein the electronic image sensor 105 is exposed by the lens apparatus 102 and 
generates a live display to the user. The preview mode variable 132 is set by a user, 
and may change in response to a press of an input button, sliding of a lens cover 
door, etc. The preview mode allows the user to set up a shot in real time. When the 
user is satisfied that the image is ready for capture, the user may initiate the image 
capture. Therefore, the preview mode is essentially continuously capturing an image 
in the electronic image sensor 105. 

[0025] The dark frame preview timer 1 38 may be used by the processor 1 09 to 
capture dark frames even when the Image capturing device 100 is in a preview 
mode. The dark frame preview timer 138 may be loaded with a predetermined time 
period, and a dark frame may be captured when the dark frame preview timer 138 
expires. The dark frame preview timer 138 may then be reloaded and the process 
repeated for as long as the image capturing device 100 is in the preview mode. 
[0026] FIG. 2 is a flowchart 200 of a noise reduction method according to one 
embodiment of the invention. In step 202, a dark frame is captured. This dark frame 
may be captured at any time and is done on a pixel-by-pixel basis. The dark frame 
capturing is substantially continuously performed, including performing a periodic 
dark frame capturing. 
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[0027] In step 206, the dark frame is stored in tiie dark frame buffer 1 23. The 
dark frame buffer 123 may store one or more dark frames, with the newest dark 
frame replacing a dark frame in the dark frame buffer 123. If the dark frame buffer 
123 can store more than one dark frame, the oldest dark frame in the dark frame 
buffer 123 may be replaced by the, newest dark frame. 

[0028] In step 21 1 , the image capturing device 100 determines whether it is 
performing an image capture. If it is not performing an image capture, the method 
branches back to step 202 where another dark frame is captured and stored. This 
branching implements the substantially continuous dark frame capturing. If an image 
capture is occurring, the method proceeds to step 218 (the image capture status 
may be determined by inspecting the capture mode variable 1 36 in the memory 

% [0029] In step 21 8, the newest {i.e., the most recently captured) dark frame is 

" subtracted from the captured image. The captured image may then be further 

JJl processed or stored as desired. 

[0030] It should be understood that the flowchart 200 shows a noise 
compensation method performed when the image capturing device 1 00 is not in a 
preview mode. The preview mode is addressed below, in the discussion 
accompanying FIG. 3. 

[0031] FIG. 3 is a flowchart 300 of a noise reduction method according to 

another embodiment of the invention. In step 304, the method determines whether 

the image capturing device 100 is in a preview mode. If it is not, the method 

proceeds to step 306; otherwise, it proceeds to step 322. 

[0032] In step 306, as before, a dark frame is substantially continuously 

captured. 
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[0033] In step 31 0, as before, the dark frame is stored. 
[0034] In step 313, as before, the method checks whether an image capture is 
occurring; if not, the method proceeds back to step 304; otherwise, it proceeds to 
step 317. 

[0035] In step 31 7, as before, the dark frame is subtracted from the captured 
image in order to perform the noise reduction. 

[0036] In step 322, in the preview mode, the image capturing device 100 checks 
the dark frame preview timer 1 38. If the dark frame preview timer 1 38 has not 
expired, the method loops and waits; othen^/ise, it proceeds to step 326. 
[0037] In step 326, the preview mode is stopped due to the expiration of the 
dark frame preview timer 138. The preview mode is stopped by momentarily closing 
the shutter (/.e., interrupting the preview mode). Thereafter, in steps 306 and 310, a 
dark frame is captured and stored, as previously discussed. The dark frame data 
therefore is captured at predetermined time intervals during the preview mode. The 
dark frame capture cannot occur until the preview mode is stopped and the shutter 
(not shown) is closed. The preview mode may then be resumed by re-opening the 
shutter. This will not objectionably affect the preview mode, as the dark frame 
capture typically takes only about 25 to about 50 milliseconds to accomplish. 
[0038] It should be understood that alternatively if the image capturing device 
100 is in the preview mode, the dark frame capture may simply be ignored until the 
preview mode is over (potentially eliminating steps 322 and 326). 
[0039] Noise reduction of the invention applies to digital still cameras and video 
cameras, including still cameras having a preview feature wherein the user may 
preview an image before capturing it. Moreover, the invention may apply to any 
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image capturing device that employs an electronic image sensor wfierein It is desired 
to reduce the noise in the signal captured by the electronic image sensor. 
[0040] The invention differs from the prior art in that it provides a highly effective 
noise compensation, but yet without interfering with the image capturing process. 
The invention accomplishes this by capturing dark frames during idle times in the 
image capturing device 100 and uses the most current dark frame data that is 
captured during idle times. This may include periodically capturing dark frames 
during a preview mode. Therefore, the invention accomplishes a highly accurate 
noise characterization and reduction, but yet while causing less interference with 
picture taking time by not consuming excess image capturing and processing time 
between shots. Therefore, the invention advantageously moves the noise 
compensation processing time away from the time between shots and accomplishes 
the dark frame sampling and noise compensation at times when the camera is 
relatively idle. 

[0041] The invention therefore provides a pixel-by-pixel noise characterization 
and reduction in order to optimally reduce noise in an image. The invention 
advantageously accomplishes this without requiring any delay in image capture and 
without a need to schedule a dark frame capture around an image capture 
occurrence or occurrences. 
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